A New Incremental Method for Function Approximation using Feed-forward Neural Networks

A sequential method for approximating vectors in Hilbert spaces called Sequential Approxi mation with Optimal Coe cients and Interacting Fre quencies SAOCIF is presented SAOCIF combines two key ideas The rst one is the optimization of the coe cients the linear part of the approximation The second one is the exibility to choose the fre quencies the non linear part The approximations de ned by SAOCIF maintain orthogonal like proper ties The theoretical results obtained prove that un der reasonable conditions the residue of the approx imation obtained with SAOCIF in the limit is the best one that can be obtained with any subset of the given set of vectors In the particular case of L it can be applied to approximations by algebraic polynomi als Fourier series wavelets and feed forward neural networks among others Also a particular algorithm with feed forward neural networks is presented The method combines the locality of sequential approxi mations where only one frequency is found at every step with the globality of non sequential ones where every frequency interacts with the others Experi mental results show a very satisfactory performance